Skip to content

Bisect a performance test case with known commits - #7912

Open
tdhock wants to merge 40 commits into
masterfrom
bisect-7687
Open

tdhock wants to merge 40 commits into
masterfrom
bisect-7687

Conversation

@tdhock

@tdhock tdhock commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

This PR adds historical commits using new git bisect functionality in tdhock/atime#131

Like #7688 but more automated.

Visualization of results here https://tdhock.github.io/2026-10-13-bisect-performance-pr7912/ shows that #4491 also slowed down this test case (not yet documented in atime test case), in addition to #4164 (already documented as the Regression commit in atime test case).
image

@tdhock tdhock changed the title Bisect 7687 Bisect a performance test case with known commits Oct 9, 2026
@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
  • HEAD=bisect-7687 much faster for DT[by,verbose=TRUE] improved in #6296

(Plot image upload to picrd.com failed; download the artifact below to view plots)

Generated via commit 1191afa

Download link for the artifact containing the test results: ↓ atime-results.zip

Task Duration
R setup and installing dependencies 6 minutes and 13 seconds
Installing different package versions 17 seconds
Running and plotting the test cases 4 minutes and 32 seconds

@tdhock tdhock added the atime Requests related to adding/improving/monitoring performance regression tests via atime. label Oct 24, 2026
Comment thread .ci/atime/tests.R
Comment on lines +337 to +347
bisect.commits=c(
Before4164 = "166e0bab317e9d9f4f1844997a315e6482ab5fd9",#a few click of parent up from first commit in PR4164 gets to https://github.com/Rdatatable/data.table/commit/166e0bab317e9d9f4f1844997a315e6482ab5fd9
PR4164="4aadde8f5a51cd7c8f3889964e7280432ec65bbc", # Merge commit of the PR that causes the regression.
Parent4558="6f360be0b2a6cf425f6df751ca9a99ec5d35ed93", #parent of PR4558 (https://github.com/Rdatatable/data.table/commit/ba32f3cba38ec270587e395f6e6c26a80be36be6)
PR4558 = "ba32f3cba38ec270587e395f6e6c26a80be36be6", # Merge commit of the PR (https://github.com/Rdatatable/data.table/pull/4558) that fixes the regression
Parent4491="3fa8b20435d33b3d4b5c26fd9b0ac14c10b98800", # PR4491 is bisect result of PR4164 and Parent5463.
PR4491="85adf09e3463838d547977ae9bc75e3b37f9cbaf",
Parent5463="052f8da6", # PR5463 is bisect result of PR4164 and Parent7401.
PR5463="19b7866112614db53eb3e909c097407d91cd6738",
Parent7401="20b463c9592c07d05ad2e89a25f84f0b998d6154", # master parent of the PR7401 which caused the recent speedup.
Fast="a3eeb4c0caec51afd66bdf9bbc16817e12a47a40"), #master 10 Sept 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these commits can be used with atime::bisect() but do not appear in the CI results:
image

@tdhock
tdhock marked this pull request as ready for review October 24, 2026 14:17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

atime Requests related to adding/improving/monitoring performance regression tests via atime.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant